GEOMESA-3582 Trino - Removes BBOX predicates from datastore SQL, adds as feature flag to the connector#3589
Merged
Conversation
elahrvivaz
reviewed
Jul 20, 2026
cwdobbins
marked this pull request as draft
July 21, 2026 12:20
…ng datastore SQL as limited to ST_* predicates.
…uiting logic to the plugin connector, which shortcuts the more expensive WKB decode during row-filtering, provides additional performance improvement for coarsely-pruned datasets.
… bounding box coordinates of the envelope, which boosts non-point, non-rectangular geometry performance
cwdobbins
marked this pull request as ready for review
July 22, 2026 03:22
cwdobbins
force-pushed
the
feature/sql-refactor
branch
from
July 22, 2026 03:35
972e4b6 to
df8976a
Compare
elahrvivaz
reviewed
Jul 22, 2026
elahrvivaz
approved these changes
Jul 22, 2026
elahrvivaz
left a comment
Contributor
There was a problem hiding this comment.
just had a question about case-sensitivity of function names, but otherwise looks good! (although I don't know the internals of trino well enough to spot any issues)
…ovided case-insensitive matching
Contributor
|
closes #3582 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GEOMESA-3582: consolidate spatial pushdown in the connector
Splits spatial-filter handling cleanly between the two
geomesa-trinomodules and makes query plans deterministic regardless of how a query arrives.Background
Previously the datastore rewrote CQL spatial predicates into composite SQL —
__X_bbox__overlap prefilters plusCASE WHENrow-level shortcuts — and thespatial_icebergconnector independently derived the same pruning from eitherST_*calls or the recognized bbox-comparison pattern. The same logical query could therefore produce different plans depending on whether it came through the datastore or as hand-written Trino SQL, and near-identical logic was maintained in two places.1. Datastore emits pure
ST_*(commit 1)TrinoFilterToSQLnow emits only row-levelST_*function calls (plus temporal / attribute predicates). Swapped operand forms are normalized column-first via the OGC complement (within(a,b) ⇔ contains(b,a)) so the connector'sST_*(ST_GeomFromBinary(col), literal)recognizer always matches. The connector is now the single place bbox/Z2/XZ2 pushdown is derived, so datastore queries and hand-written Trino SQL get identical treatment and deterministic plans.2. Connector-side bbox short-circuit for rectangle
ST_Intersects(commit 2, feature-flagged)Emitting pure
ST_*removes the datastore's inlineCASE WHENrow-level shortcut, which had let a row whose bbox was fully inside the query rectangle skip the WKB decode. This commit recovers a sound version of that saving entirely inside the connector, with no datastore coupling. pruning; those domains ride to the worker in the table handle'sunenforcedPredicate. For the one case the bbox proves exactly — a rectangleST_Intersectson a Z2/point geometry column —SpatialConnectorMetadataclaims the predicate enforced (replacing only thest_intersectsnode withTRUE, never theis_visiblerow filter or any other conjunct) and wraps the handle soSpatialPageSourceProviderbecomes the authoritative filter. Reading only the cheap bbox columns, the page source:intersectstest only on the thin envelope-boundary shell.For point geometry columns the stored bbox is degenerate (
xmin == xmax,ymin == ymax), so the classifier reads just the two coordinate columns and the box test collapses to a point-in-range check.st_intersectscall is claimed —is_visibleand every other conjunct still execute.geomesa.spatial.bbox-page-filter(on by default); see commit 3 for the flag's cost/benefit and how to disable it.3. Connector-side bbox reject pre-filter for the general case (commit 3)
Commit 2's short-circuit engages only for the exactly-provable case (rectangle
ST_Intersectson a Z2/point column). This commit extends the same page source with a reject-only mode covering every other spatial predicate the connector could not claim enforced — non-rectangular query geometries, non-point columns,ST_Contains, the outer envelope of aDWITHINdistance query, etc.In reject-only mode the page source drops any row whose bbox cannot overlap the query envelope (via
SourcePage.selectPositions, before the geometry column is materialized) and hands the survivors to the engine's exactST_*residual unchanged. It reuses the same reject-bound machinery as the short-circuit's reject pass. This is a pure pre-filter:icebergcatalog (no pushdown) across rectangle, triangle,ST_Within,DWITHIN, and non-spatial filters.Both the short-circuit (commit 2) and this reject pre-filter are controlled by the single
geomesa.spatial.bbox-page-filtercatalog property, on by default. Its value is inversely proportional to spatial partition quality: on a well-truncate-partitioned Z2/XZ2 table, file/row-group pruning already removes most non-matching rows, so the extra bbox-column read can outweigh the saved decodes (most visibly forDWITHIN, which must materialize the geometry for the exact distance test regardless); on a coarsely-partitioned table the row-level rejection catches what file pruning misses. Setgeomesa.spatial.bbox-page-filter=falseto fall back to partition + per-file stat pruning plus the engine's exactST_*on every surviving row.